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A VISUALLY-COUPLED  AIRBORNE  SYSTEMS  SIMULATOR  (VCASS)  - 
AN  APPROACH  TO  VISUAL  SIMULATION 


DEAN  F.  KOCIAN 

6570  Aerospace  Medical  Research  Laboratory 


In  recent  years  Air  Force  operational  units  have  experienced  a 
continuing  trend  downward  in  the  number  of  flight  hours  in  aircraft 
that  can  be  provided  to  each  individual  pilot  for  training  and  main- 
taining proficiency.  This  comes  at  a time  when  aircraft  systems  are 
becoming  ever  more  complex  and  sophisticated  requiring  comparatively 
more  hours  for  training  to  maintain  the  same  relative  flying  proficiency. 
With  increasing  costs  for  fuel  and  aircraft  and  the  failure  of  DoD  fund- 
ing to  keep  pace  with  these  costs,  the  trend  is  almost  sure  to  continue. 
In  adjusting  to  the  realities  of  keeping  overall  experience  at  a satis- 
factory level  and  reducing  costs,  procurement  of  aircraft  simulators 
has  become  a necessity. 

The  rapid  proliferation  of  simulators  with  no  standard  technical 
criteria  as  a guide  has  resulted  in  the  evolution  of  several  different 
design  approaches.  Most  existing  visual  scene  simulators  utilize 
electro-optical  devices  which  project  video  imagery  (generated  from  a 
sensor  scan  of  a terrain  board  or  a computer  generated  imagery  capabil- 
ity) onto  a hemispherical  dome  or  set  of  large  adjacent  CRT  displays 
arranged  in  optical  mosaics  with  the  weapon,  vehicle,  and  threat 
dynamics  being  provided  by  additional  computer  capabilities. 

These  large  fixed-base  simulators  suffer  from  the  following  draw- 
backs. The  majority  of  the  visual  projection  techniques  used  in  these 
simulators  do  not  incorporate  infinity  optics  which  provide  collimated 
visual  scenes  to  the  operator.  Those  which  do  are  large  and  expensive 
and  incorporate  large  CRT  displays.  The  luminance  levels  and  resolution 
of  these  displays  are  usually  low  and  do  not  represent  true  ambient 
conditions  in  the  real  environment.  Additionally,  hemispherical  infinity 
optics  are  difficult  to  implement  and  this  technique  requires  excessive 
computer  capacity  to  generate  imagery  due  to  the  need  for  refreshing  an 
entire  hemisphere  instantaneously,  regardless  of  where  the  crew  member 
is  looking.  In  this  regard,  existing  computer  capability  is  not  used 
effectively  to  match  the  channel  capacity  of  the  human  visual  system. 
There  are  also  generally  no  stereoscopic  depth  cues  provided  for  outside- 
of-cockpit  scenes.  Another  important  drawback  to  these  simulators  is 
that  the  visual  simulation  is  not  transferrable  to  the  actual  flight 
environment,  i.e.,  the  ground-based  system  cannot  be  transferred  to  an 
actual  aircraft  to  determine  simulation  validity.  Finally,  most 
existing  techniques  are  very  expensive  and  do  not  allow  the  flexibility 
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of  incorporating  other  display  design  factors  such  as  different  head-up 
display  image  formats,  fields-of-view  (FOV),  representative  cockpit 
visibilities,  and  optional  control  and  display  interfaces. 

t -><.  « * S CX  nfit a)-~  v. 

A quite  different^approach  to  solving  the  visual  presentation  _ „ , 
problems  of  aircraft  simulators  is  to  employ  the  use  of,  visually 
coupled  systems  (VCS).  For  many  years  it  has  been  the  mission  of  thef  t-'  • 
Aerospace  Medical  Research  Laboratory  to  optimize  the  visual  interface 
of  crew  members  to  advanced  weapon  systems.  This  mission  has  been  pri- 
marily pursued  in  two  areas:  (1)  the  establishment  of  control/display 

engineering  criteria;  and  (2)  the  prototyping  of  advanced  concepts  for 
control  and  display  interface.  An  important  part  of  fulfilling  this 
mission  has  been  the  development  of  VCS  components  which  includes  head 
position  sensing  systems  or  helmet  mounted  sights  (HMS),  eye  position 
sensing  systems  (EPS)  and  helmet  mounted  displays  (HMD).  -r  . 
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In  the  process  of  accomplishing  this  work,  it  has  been  ascertained 
that  many  of  the  current  Air  Force  air-to-air  and  air-to-ground  weapon 
systems  problems  can  be  related  to  deficiencies  in  the  configurations 
of  control  and  display  components  which  interface  the  crew  member  to 
aircraft  fire  control,  navigation,  flight  control  and  weapon  delivery 
subsystems.  These  interfaces  tend  to  either  overly  task  load  the  crew 
member  or  prevent  optimum  utilization  of  innate  visual,  perceptual  and 
motor  capabilities.  These  limitations  are  especially  apparent  in  fire 
control  and  weapon  delivery  applications  where  visual  target  acquisition 
and  weapon  aiming  are  required  along  with  primary  piloting  tasks.  Under 
high  threat  conditions,  the  flight  profiles  necessary  for  survivability, 
as  well  as  mission  success,  dictate  that  all  essential  tasks  be  performed 
effectively,  accurately  and  most  important  expediently.  With  the  recent 
advent  of  advanced  digital  avionics  systems,  the  control  and  display 
design  issue  is  further  complicated.  The  proliferation  of  dedicated 
control  and  display  subsystems  in  current  aircraft  cockpits  has  neces- 
sitated the  development  of  multi -mode  displays  and  control  input  devices. 
In  addition,  more  exotic  virtual  image  display  devices  (head-up  display/ 
helmet-mounted  display)  and  unique  control  devices  such  as  the  multi- 
function keyboard,  helmet-mounted  sight  and  fly-by-wire  subsystems  have 
appeared.  In  this  regard,  the  design  options  open  to  the  avionics  as 
well  as  control  and  display  designer  are  great,  thereby  generating  a 
real  need  for  human  engineering  design  criteria  to  elucidate  the  image 
quality  characteristics,  information  formatting  and  interface  dynamics 
which  optimize  the  operator  interface  with  these  advanced  systems. 

The  process  of  establishing  practical  design  criteria  with  the 
number  of  options  that  are  available  is  a laborious  and  time  consuming 
task,  especially  if  validation  in  flight  environments  becomes  necessary. 
Typically,  flight  testing  is  very  expensive  and  does  not  allow  flexi- 
bility as  well  as  consistent  replication  of  experimental  conditions. 


Due  to  these  factors,  high  fidelity  ground-based  simulation  is  the  only 
realistic  alternative.  However,  it  now  becomes  necessary  to  develop 
simulation  methodology,  techniques  and  apparatus  which  are  subject  to 
flight  test  validation.  It  is  feltifchat  the  unique  capabilities  of  a 
visually-coupled  system  (VCS  - combination  of  a helmet-mounted  sight 
and  helmet-mounted  display)  can  meet  the  simulation  requirements^stated 
above  as  well  as  improve  upon  existing  ground  based  simulation  techniques, 
described  earlier.  It  is  out  of  this  thinking  that  the  VCASS  concept 
evolved. 

A more  detailed  analysis  of  the  problem  has  produced  a set  of 
characteristics  which  a more  ideal  aircraft  simulator  might  possess.  Of 
primary  importance  is  that  it  should  be  a flexible  visual  scene  simula- 
tion providing  synthesized  out-of-the-cockpit  visual  scenes  and  targets, 
a representative  vehicle  whose  type  can  be  altered,  threat  and  weapon 
dynamics,  flexibility  of  control  and  display  configurations,  and  inputs 
from  sensor  or  real  world  imagery.  It  should  be  portable  if  possible 
and  provide  alternatives  for  crew  station  display  options  including 
number  and  configuration.  This  simulator  should  also  be  useable  in  both 
simulated  air-to-ground  weapon  delivery  and  air-to-air  engagement 
scenarios.  Finally,  it  should  be  possible  to  use  the  same  system  in 
ground  fixed  base  and  motion  base  simulators  as  well  as  in  aircraft. 

As  an  approach  to  meeting  these  requirements  the  VCASS  concept  and 
program  was  initiated.  Its  objective  is  to  develop  and  demonstrate  a 
self-contained  airborne  and  ground-based  man-in-the-loop  visual  simulator 
for  the  engineering  of  advanced  weapon  systems.  The  approach  that  will 
be  followed  to  obtain  this  objective  will  be  to  integrate  VCS  hardware 
with  state-of-the-art  computer  image  generators  to  provide  a synthesized 
hemispherical  visual  space  that  will  display  target  and  environmental 
images.  Included  in  this  approach  is  the  use  of  real  and/or  simulated 
plant  dynamics. 

The -key  components  of  VCASS  will  be  VCS  hardware  which  includes 
the  HMS  and  HMD.  These  components  are  used  to  "visual ly-couple"  the 
operator  to  the  other  system  components  he  is  using.  AMRL  has  pioneered 
efforts  in  the  research,  development  and  testing  of  these  hardware 
techniques. 

Specifically,  the  concept  of  the  VCASS  is  to  utilize  the  HMS  as 
a means  of  selecting  information  within  a synthesized  visual  space  and 
to  use  the  helmet  display  as  the  visual  input  device  for  presenting  that 
information  to  the  operator  as  a collimated  virtual  image.  This  allows 
head-up  display  type  symbology  and/or  imagery  to  be  generated  to  repre- 
sent a full  hemisphere,  out-of-the-cockpit  view,  a portion  of  which  the 
operator  perceives  on  the  helmet  display.  The  scale  or  size  of  this 
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instantaneous  portion  of  the  total  field  is  a function  of  the  field-of- 
view  of  the  HMD.  The  orientation  of  the  instantaneous  field-of-view  is 
determined  and  selected  in  accordance  with  head  orientation  as  measured 
by  the  HMS.  In  other  words,  if  the  field-of-view  of  the  HMD  is  30 
degrees  the  observer  sees  a 30  degree  instantaneous  view  of  a hemis- 
pherical digital  symbol  set.  This  instantaneous  view  moves  in  a one- 
to-one  correspondence  with  head  movement.  In  essence,  the  total  hemis- 
pherical scene  is  available  to  the  operator  a field-of-view  at  a time. 

A system  diagram  and  pictoral  of  the  functional  elements  required 
to  accomplish  the  VCASS  are  depicted  in  Figure  1.  The  operator  utilizes 
conventional  control  devices  (control  stick,  throttle,  rudder  pedals, 
etc.)  to  input  a digital  computer  which  provides  the  manipulation  of  the 
vehicle,  weapon  and  threat  states  as  a function  of  preprogrammed  dynamic 
characteristics.  This  information  is  then  used  to  manipulate  synthesized 
symbology  and  imagery  in  terms  of  orientation,  scale,  target  location, 
etc.  as  a function  of  the  plant  state.  A representative  visual  scene 
generated  by  the  graphics  or  sensor  imagery  generators  is  selected  by 
the  operator  line-of-sight  orientation  as  measured  by  the  helmet-mounted 
sight.  Again,  the  amount  of  information  selected  is  governed  by  the 
instantaneous  field-of-view  of  the  helmet-mounted  display  (typically 
30  degrees  to  40  degrees).  The  helmet  display  electronics  receives  the 
selected  portion  of  the  symbology  and  sensor  information  and  displays 
the  video  imagery  to  the  operator  through  the  helmet  display  optics  in 
the  proper  orientation  within  three-dimensional  space.  For  an  airborne 
VCASS  capability,  it  is  only  necessary  to  install  the  VCS  components 
along  with  a small  airborne  general  purpose  computer  in  a suitable  air- 
craft and  interface  a representative  programmable  symbol  generator  to  an 
on-board  attitude  reference  system  in  order  to  synthesize  either  airborne 
or  ground  targets.  This  approach  has  the  ultimate  flexibility  of  utiliz- 
ing the  same  symbol  set,  threat  dynamics,  etc.,  in  the  air  that  were 
originally  used  in  the  ground  simulation.  In  either  case,  the  crew 
member  will  engage  electronic  targets  (either  air-to-air  or  air-to- 
ground)  and  launch  electronic  weapons.  His  performance  in  these  tasks 
in  turn  will  be  recorded  and  assessed  for  performance  or  utilized  as 
training  aids  for  the  crew  member  or  operator. 

Figure  2 depicts  a more  advanced  configuration  of  the  helmet- 
mounted  sight  and  display  that  will  be  used  in  the  VCASS  installation. 

The  helmet-mounted  sight  and  display  are  integrated  into  one  compact 
unit  that  allows  a prealigned  visually-coupled  system  package  to  be 
easily  connected  and  disconnected  from  a standard  flight  helmet.  The 
helmet-mounted  sight  transducers  represented  by  the  STA  and  SRAH  are 
small  and  compact  and  allow  a more  or  less  benign  mounting  in  the  air- 
craft cockpit.  The  side  mounted  helmet-mounted  display  is  capable  of 
at  least  a sixty  degree  field-of-view  in  this  configuration  as  compared 
to  30  to  40  degrees  for  a visor  display  with  a reasonable  form  factor. 
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Compared  to  other  simulation  systems  this  configuration  permits  a rela- 
tively easy  transition  from  the  ground  to  airborne  environment  for 
feasibility  studies  and  demonstrations. 
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The  VCASS  concept  of  simulation  provides  a method  of  artificially 
duplicating  all  the  standard  scenarios  that  are  provided  by  more  con- 
ventional simulators  plus  more.  For  air-to-air  formats  the  simulation 
can  take  the  form  of  programmed  maneuvers  as  a function  of  time,  evasive 
maneuvers  based  on  a set  of  computer  algorithms  that  permit  an  adaptive 
strategy  for  the  target,  or  a totally  competitive  simulation  where  the 
instructor  maneuvers  the  target.  For  air-to-ground  formats  the  target 
or  threat  can  be  stabilized  at  prestored  ground  coordinates,  surviv- 
ability against  an  active  threat  can  be  tested,  and  target  size  and 
vulnerability  can  be  varied.  Additionally,  visual  display  design  cri- 
teria can  be  developed  for  fixed  base,  moving  base,  and  airborne  type 
simulators  to  investigate  and  enhance  techniques  for  simulation  optimi- 
zation. Finally,  prototype  visual  display  configurations  in  virtual 
space  can  be  developed  and  altered  by  simply  changing  the  related  soft- 
ware. 

The  cost/performance  advantages  of  the  VCASS  concept  as  depicted 
above  appear  to  be  numerous  and  worthwhile.  Of  primary  importance  is 
the  fact  that  a full  hemisphere  of  collimated  visual  information  can  be 
provided  which  depends  solely  on  the  head  orientation  limits  of  the  user. 
This  hemisphere  of  synthesized  visual  target  and  environmental  images 
can  be  accomplished  without  the  need  for  costly  domes  or  fixed  mosaic 
infinity  optics.  Conservation  of  computer  capacity  is  provided  as  a 
result  of  necessitating  only  the  small  instantaneous  field-of-view  of 
the  HMD  to  be  provided  to  the  operator.  This  suggests  that  it  should 
be  possible  to  use  conventional  general  purpose  computers  for  computing 
and  creating  the  environment,  vehicle  threat  and  weapon  plant  dynamics 
as  well  as  to  control  a small  special  purpose  symbology  generator.  The 
image  quality  should  be  very  high  at  the  greater  luminance  levels  and 
color  and  stereo  capabilities  are  also  possible.  Also,  all  threat  air- 
craft and  weapon  dynamics  are  programmable  providing  an  ultimate  flex- 
ibility in  design  parameters  and  the  cockpit  display  (HUD  symbology  sets) 
can  be  manipulated  easily  to  determine  the  interaction  between  the 
symbol  sets  and  the  synthesized  real  world  imagery.  Finally,  almost 
all  components  including  the  most  critical  ones  can  be  utilized  in  either 
a ground-based  or  airborne  simulator. 

If  all  the  critical  components  were  in  an  ideal  form  for  the 
VCASS  application  it  would  merely  require  that  one  perform  the  hardware 
interface,  software  development,  and  test  the  performance  obtained  out 
of  the  final  system  configuration.  However,  VCS  hardware  development 
and  performance  has  lagged  somewhat  relative  to  the  performance  capabil- 
ities of  other  components  that  are  to  be  used  in  the  VCASS  simulation. 
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Added  to  this  is  the  fact  that  the  VCASS  simulation  imposes  certain 
psychophysical  considerations  on  the  entire  system  configuration. 

Among  the  most  important  of  these  is  the  required  instantaneous  field- 
of-view  of  the  helmet-mounted  display  beyond  which  there  will  be  rela- 
tively little  improvement  in  operator  performance  when  flying  the 
VCASS  system.  The  important  decisions  to  be  made  here  are  the  amount 
of  area  on  the  display  that  must  have  a high  resolution  format  and  how 
large  the  display  field-of-view  must  be  to  provide  necessary  informa- 
tion cues  in  the  peripheral  vision.  Another  important  requirement  is 
to  determine  the  required  update  rates  and  throughput  delays  to  be 
allowed  in  the  head  position  sensing  information  in  order  to  minimize 
perceptable  lags  in  the  change  of  information  on  the  helmet-mounted 
display.  The  symbology  and  environmental  information  presented  on  the 
display  must  also  change  realistically  in  relation  to  changes  in  observer 
look  angle  and  aircraft  parameters  in  a manner  that  appears  natural  with 
no  confusing  contradictions.  The  crew  member  must  be  able  to  relate  to 
aircraft  attitude  and  heading  at  any  look  angle.  Experience  already 
gained  on  an  interim  VCASS  configuration  has  shown  that  these  require- 
ments will  necessitate  a major  symbology  and  format  design  effort. 

Some  of  the  above  mentioned  areas  of  consideration  must  wait  for 
further  testing  before  a design  approach  can  be  formulated  while  others 
will  not.  To  some  extent  the  maximum  obtainable  performance  of  certain 
parameters  of  the  most  suitable  VCS  components  is  already  known  and  must 
be  accepted  or  its  effects  reduced  by  changes  to  other  portions  of  the 
VCASS  system.  For  the  helmet-mounted  sight  the  individual  added  require- 
ments are  both  more  easily  defined  and  met  than  is  the  case  for  the 
helmet-mounted  display. 

Even  though  individual  requirements  for  the  helmet-mounted  sight 
are  straightforward  in  an  engineering  sense  the  total  design  change 
package  represents  a significant  increase  in  performance  over  systems 
currently  available.  To  minimize  perceptual  lags  and  prevent  loss  of 
head  movement  coverage,  the  update  rate  must  be  increased  from  the 
presently  available  33Hz  to  100Hz  or  more  and  the  motion  box  must  be 
enlarged  from  one  to  four  cubic  feet.  In  order  to  provide  sufficient 
information  to  simulate  the  parallax  of  aircraft  structures  on  the 
helmet-mounted  display  as  the  operator  moves  his  head,  a six-degree- 
of-freedom  HMS  is  required  that  provides  not  only  attitude  information 
(azimuth,  elevation  and  roll)  but  x,  y,  z position  information  as  well. 
Another  significant  problem  is  the  smallest  change  in  head  movement 
which  can  be  measured  by  the  HMS  and  therefore  provide  updated  informa- 
tion for  changing  the  video  imagery  on  the  helmet-mounted  display.  Pre- 
liminary studies  have  suggested  that  resolution  must  be  increased  from 
0.097  to  0.03  degrees  to  eliminate  noticeable  step  changes  in  the 
display  presentation  as  perceived  by  the  observer.  Finally,  some  form 
of  output  stabilization  must  be  provided  to  reduce  head  jitter  noise 
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from  whatever  source  to  an  acceptable  level  that  would  not  visibly  de- 
grade display  resolution. 

The  design  considerations  involved  in  building  a helmet-mounted 
display  for  the  VCASS  simulation  present  a more  formidable  and  subjective 
set  of  problems  whose  solution  is  not  entirely  clear.  It  is  certain  that 
a larger  display  field-of-view  is  required  but  how  large  remains  an 
unanswered  question.  The  optical  physics  that  are  part  of  the  display 
design  imposed  constraints  which  are  difficult  to  resolve.  Currently, 
an  interim  display  possessing  a 60  degree  instantaneous  field-of-view 
is  planned  for  the  VCASS;  however,  recent  studies  have  shown  that  this 
may  not  be  large  enough  especially  when  viewed  with  one  eye.  This  leads 
naturally  to  biocular  or  binocular  configurations.  A whole  host  of  human 
factors  problems  then  becomes  important  including  brightness  disparity, 
display  registration,  and  eye  dominance.  The  decision  whether  or  not  to 
include  color  also  becomes  a major  design  decision  not  only  because  of 
the  engineering  development  required  but  because  user  acceptance  may 
weigh  heavily  on  this  factor. 

If  the  design  problems  can  be  overcome  it  appears  that  the  benefits 
of  the  VCASS  for  training  are  great.  Experience  for  the  crew  member  can 
be  provided  in  many  aircraft  types  against  a wide  variety  of  threats, 
armament,  encounter  dynamics,  etc.  Feedback  in  the  training  situation 
can  be  significant  and  rapid  with  optional  instructor  involvement,  repe- 
tition and  instant  replay  on  all  encounters,  and  the  fact  that  an  air- 
borne vehicle  can  use  VCASS  components  to  correlate  ground-based  results. 
The  cost  effectiveness  of  this  approach  seems  to  be  overwhelming.  The 
cost  of  this  system  is  assured  of  being  significantly  less  than  the 
costly  ground  visual  simulators  now  in  existance.  One  system  can  be 
used  for  the  air  and  ground  environment.  In  the  airborne  case  no  darts, 
drones,  chase  planes  or  bombing  ranges  are  required  and  no  aircraft  arma- 
ment installation  or  expenditure  of  munitions  is  needed. 


